The present disclosure relates to the field of information retrieval, and specifically to the field of information retrieval based on a query by a user. Still more specifically, the present disclosure relates to the field of interpreting user-generated queries.
Many mobile devices today have the capability of receiving an oral question from a user, and then responding with an answer using speech synthesis.
For example, U.S. Patent Application Publication No. 2012/0016678 describes an intelligent automated assistant that engages with a user using a natural language dialog in order to answer specific questions from the user. However, such systems are limited to speech inputs from the user, which can result in erroneous responses by the mobile device. For example, a user may ask a question that is either incorrectly interpreted by the system, or the system simply does not understand the question at all. This requires the user to follow-up with additional voice commands, clarifications, etc.
Other devices provide information about non-verbal (i.e., non-speech) inputs, but do not allow the user to generate and ask questions about the non-verbal inputs.
For example, U.S. Patent Application Publication No. 2012/0124461 describes a technique for annotating street level images with contextual information overlaid on the images. Similarly, U.S. Patent Application Publication No. 2013/0346068 describes a system that tags a digital photograph with certain user-provided terms. However, this type of art is limited to auto-populating an image with predefined information, without regard to a specific question that the use may have.
Thus, the prior art, either singularly or in combination, fails to provide a system that allows a user to ask a question, and then to augment that question using a non-verbal input that clarifies the question. That is, the prior art does not provide a system that allows a user to present user-generated questions to a device about non-verbal inputs such as visual images, scents, sounds, etc. The present invention provides one or more solutions to this long-felt need.